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Quantum error-correcting codes are analyzed from an information-theoretic perspective centered 
on quantum conditional and mutual entropies. This approach parallels the description of classical 
error correction in Shannon theory, while clarifying the differences between classical and quantum 
codes. More specifically, it is shown how quantum information theory accounts for the fact that 
"redundant" information can be distributed over quantum bits even though this does not violate the 
quantum "no-cloning" theorem. Such a remarkable feature, which has no counterpart for classical 
codes, is related to the property that the ternary mutual entropy vanishes for a tripartite system in a 
pure state. This information-theoretic description of quantum coding is used to derive the quantum 
analogue of the Singleton bound on the number of logical bits that can be preserved by a code of 
fixed length which can recover a given number of errors. 



PACS numbers: 03.65.Bz,89.70.-(-c 



I. INTRODUCTION 

The potential use of quantum computers for solving 
certain classes of problems has recently received a con- 
siderable amount of attention (for a review see, e.g., 
|El-p[). A major obstacle in the building of quantum com- 
puters, however, is the coupling of the computer with 
its environment or the decoherence, which rapidly de- 
stroys the quantum superposition at the heart of quan- 
tum algorithms. An essential element in the realization of 
such quantum computers is therefore the use of quantum 
error-correcting codes, which have been shown to ensure 
protection against decoherence 10^1^1 • Quantum codes 
are similar in many respects to classical codes. In clas- 
sical coding theory, logical words (of k bits) are encoded 
into codewords (of n > fc bits). The latter are suitably 
chosen among the set of all 2" possible words of n bits so 
that the alteration because of noise of say t bits (at most) 
can be recovered. A specific set of codewords then consti- 
tutes an [n, k, t] code, encoding k bits into n bits and cor- 
recting all patterns of t (or fewer) errors among those n 
bits. The simplest example of a classical code with fc = 1, 
n = 3, and f = 1 is the repetition code where a logical 
bit (or 1) is encoded into 000 (or 111); decoding is sim- 
ply performed using the majority rule, which is enough 
to recover t — 1 errors. In classical coding theory, cor- 
rupted data is thus restored by introducing redundancy 
(n > fc), that is by duplicating part of the information 
that must be preserved. (In the above — very inefficient — 
example, information is triplicated.) In quantum coding 
theory, the central issue is to find a set of 2*^ quantum 
codewords (of n qubits) such that quantum information 
can be protected against the alteration due to coupling 
with an environment (i.e., such that the quantum system 
survives decoherence). At first sight, it seems that, since 
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the duplication of an arbitrary quantum state is forbid- 
den by the quantum no-cloning theorem |15 , "quantum 
redundancy" is impossible. However, after the pioneer- 
ing work of Shor B, it has been realized that quantum 
coding is achievable in spite of the no-cloning theorem, 
and a great deal of work has recently been devoted to 
this issue p-H . It has been shown that quantum infor- 
mation can be distributed over many qubits through a 
suitable encoding and subsequently recovered after par- 
tial alteration, without violating the no-cloning theorem. 

In this paper, we aim at clarifying some aspects of 
quantum coding from a perspective centered on quan- 
tum entropies. It has been shown recently that clas- 
sical and quantum entropies can be described within a 
unified information-theoretical framework involving neg- 
ative conditional entropies |l6| , |l7|l , as briefiy outlined in 
Appendix |A[ Here, we apply this framework to quantum 
error-correcting codes, paralleling the classical descrip- 
tion of error correction in Shannon theory. We show that, 
for an arbitrary entanglement between the logical words 
and a "reference" system to be preserved, the quantum 
mutual entropy between this "reference" and any "inter- 
acting" part of the codewords must be vanishing prior to 
decoherence. In other words, an entropic condition for 
perfect quantum error correction is that the "reference" 
system is statistically independent of any arbitrarily cho- 
sen part of the codewords that might interact with the 
environment. This condition relies on the conservation 
of quantum mutual entropies implied by unitarity, along 
with the property of strong subadditivity of quantum en- 
tropies (see Appendix H) . It expresses the fact that the 
environment cannot become directly entangled with the 
"reference" system (entanglement may arise only via the 
codewords), or, roughly speaking, that the environment 
cannot extract information about the logical words. 



We continue by deriving the analogue of the Single- 
ton bound for quantum codes ||ll[, i.e., k < n — it, us- 
ing simple arguments based on this entropic approach. 
Such an information-theoretic description of coding sheds 
new light on the interpretation of this bound in terms of 
"weak" cloning. While the quantum bits that are altered 
as a result of any error are statistically independent of 
the reference (or the encoded logical word), the quan- 
tum information stored in the entire codeword remains 
unaffected. This results from the fact that the ternary 
mutual entropy vanishes for any entangled tripartite sys- 
tem in a pure state, a property which has no classical 
counterpart |l7[ . The central point is that, in contrast 
with classical codes, no duplicating — or full cloning — is 
achieved by quantum error-correcting codes. Rather, a 
"weak" quantum cloning is achieved, such that any part 
of the codeword susceptible to decohere appears inde- 
pendent of the reference although the entire codeword 
remains entangled with it. This purely quantum situ- 
ation is forbidden in classical information theory due to 
the non-negativity of Shannon conditional entropies, and 
reflects a fundamental difference between classical and 
quantum error correcting codes. 



II. QUANTUM ERROR CORRECTION 

Let us consider a set of orthogonal logical states |il) 
(with i — l,---2*'') which are encoded into orthogonal 
codewords |zq) consisting of n qubits. (The index Q 
refers to the quantum channel on which the codewords 
are sent.) The states ji^) belong to the logical Hilbert 
space Hl of dimension cIl = 2*^ spanned by the k logi- 
cal qubits, while the states jig) belong to Tig of dimen- 
sion dq — 2". We have clearly dg > d^, which is the 
quantum equivalent of classical "redundancy" : the logi- 
cal states are encoded in some 2'^-dimensional subspace 
of the full 2"-dimensional Hilbert space so that part of 
the information in the n qubits is "redundant" . Qualita- 
tively speaking, n — k qubits of the codewords represent 
redundant information (they are equivalent to the "check 
bits" of classical codes |1^). In Section IV, we will make 
this concept of quantum "redundancy" more quantita- 
tive. 

The key property of a quantum code lies in its abil- 
ity to protect an arbitrary superposition of logical states 
^Oijii) against decoherence. Equivalently, a quantum 
code is such that the entanglement of the k logical qubits 
with a "reference" system R is preserved against deco- 
herence. In fact, this description of quantum coding as 
a mean to transmit (or conserve) entanglement with re- 
spect to R in spite of the interaction with an environment 
is more convenient for our information-theoretic descrip- 
tion and will be adopted in the following. Accordingly, 
we start by considering the initial entangled state 
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(2.1) 



where R and L refer to the reference and logical states, 
respectively. (This is the Schmidt decomposition of a 
pure entangled state.) We then consider the transforma- 
tion of \iI]rl) due to encoding followed by decoherence. 
Encoding is performed by use of a unitary transformation 
that maps the states |jl)|0) to the codewords jig), where 
|0) stands for the initial state of the n^k auxiliary qubits 
(or check bits). Thus, after encoding, the joint state of 
the reference R and the quantum channel Q is 



\^Rq) =^o.iVR)VQ) 



(2.2) 



It is a pure state of vanishing entropy S{RQ) = 0; the 
quantum entropies of i? and Q arc S{R) ~ S{Q) = H[ai\, 
where H stands for the Shannon entropy. 



i/[a,] = -^|a,plog| 



(2.3) 



Let us suppose now that the codewords are sent on a 
noisy quantum channel in which they suffer decoherence 
due to an environment E. Following Schumacher's model 
of a noisy channel |19[| , we assume that the environment is 
initially in the pure state |0) and then interacts with the 
channel according to the unitary transformation Uqe , so 
that the joint state of the entire system becomes 

l^WQ'E') = ilR®UQE)Y.a^^iR)\iQm (2.4) 

j=l 

(The prime refers to the systems a/ter decoherence.) This 
noisy channel is pictured in Fig. ^ and will be the basis of 
our description of quantum coding in terms of quantum 
entropies. More specifically, we will consider a "deter- 
ministic" error model in which the position of the erro- 
neous bits is known, usually referred to as the quantum 
erasure channel [13| . In this channel, the decoherence in- 
duced by the environment involves e qubits at known lo- 
cations, i.e., e erasures. The component Qe (of e qubits) 
of the codeword interacts with E (suffers e erasures), 
while the rest Qm (of n — e qubits) is left unchanged by 
this interaction. Accordingly, the unitary transformation 
in Eq. (^) is of the form 



Uqe - Ic 



Uq^e 



(2.5) 



As an example, we can suppose that the environment is 
made of e qubits initially in a |0) state and that Uq^e ef- 
fects the exchange between these qubits and the e qubits 
of Qe (a reversible operation). As a result, the qubits 
of Qe are erased (reset to |0)) while the qubits of E get 
the original value of the erased qubits. As the environ- 
ment is traced over in order to determine the state of 



the channel Q after decoherence, quantum information 
is apparently erased even though the overall process is 
unitary. Of course, any other Uq^e could result from 



decoherence, and a quantum erasure-correcting code will 
be such that the entanglement with R is preserved for an 
arbitrary Uq^e- 



FIG. 1. Schematic model of a noisy quantum channel preceded by encoding and followed by decoding. The logical states 
(system L of k qubits) are entangled with the reference system R. Encoding, using an ancilla A oi n — k "check" qubits 
initially in a |0) state, yields the codewords (system Q of n qubits). Then, e qubits (Qe) are "erased" by interacting with the 
environment E via Uqe, while the n — e remaining ones (Qu) are unchanged. Decoding, involving the "erased" qubits Qj. along 
with the unchanged ones Qu yields the k logical bits L in the initial entangled state i^rl with the reference R. The primes 
refer to the systems after environment-induced decoherence. 
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Before discussing coding and decoherence using quan- 
tum entropies (Section IV), let us first review some ba- 
sics of quantum error-correcting codes. It is known that, 
rather than coupling the codewords with an environment, 
one can model the errors by use of error operators E. For 
the purpose of error correction, it is enough to consider 
errors of the type Ux (bit-flip), CTz (phase-flip), and Uy 
(bit- and phase- flip), since, by linearity, a code that can 
correct these errors can correct arbitrary errors [fl. For 
a [n, fc,f] code, i.e., a code correcting t errors at most, 
the error operators E applied on the codewords are of 
the form 1®("-*) (g) £"»*, i.e., the tensor product of the 
identity on n — i qubits and t one-bit error operators on 
the altered qubits. The one-bit error operators are any 
linear combinations of the algebra basis 1 1 , fja, , CTj, , CTz}. It 
has been shown by Knill and Laflammc [|ll| that a neces- 
sary and sufficient condition on quantum error-correcting 
codes is that 



{iQ\EiE,\iQ) 
{iq\EIE,\jq) 



{jq\eIe,\jq) (2.6) 

for i =^ j (2.7) 



where the jig) and |jq) are any two codewords and Ea, 
Eh are chosen from the set of i-error operators defined 
above. Conditions ( ^.6[ ) and (2/7) can be understood by 
considering the decoding operation as an "inverse" uni- 
tary transformation p4| that maps the n qubits of the 
corrupted codeword Q' into k qubits (the original logical 
word L) and n — k check qubits (the ancilla A'), as rep- 
resented in Fig. nl Considering the action of decoding on 
two codewords I^q) and \jq) that have been corrupted by 
errors Ea or Eh, it can be shown that the state in which 
the ancilla is left cannot depend on the logical state, that 
is the decoding must be such that 



Ea\lQ) - 


■> |iL)«) \Aa 


Eb\lQ) - 


-> |jl)«) \Ah) 


EaljQ) - 


■> \Jl) ® \Aa 


Eb\jQ) - 


■> \3l) ® \Ah 



(2.8) 



In other words, the final state of A must be the same 
for both codewords |zq) and \jq), and depend only on 
the error syndrome a or b. This condition is clearly re- 
quired in order to recover an initial arbitrary superposi- 
tion ^^ aili]^) {i.e., the ancilla must be in a tensor prod- 
uct with t he k logical qubits after decoding). Conditions 
(^.6|) and (2__7) then result straightforwardly from the or- 
thogonality of the logical states Jil) and \jl), and the 
conservation of scalar products by unitarity. 

The above considerations also apply to the quantum 
erasure channel in which the position of the e erroneous 
bits is known 



|13|. Note that conditions ( |2.6| ) and < \2.l\ ) 
obviously correspond to the case where the errors are ap- 
plied at t unknown positions in the codeword. Clearly, if 
the error-correcting code aims at correcting for erasures 
only, the error operators Ea and Eb differ from each other 
by one-bit error operators at the same positions only. 
Therefore, as the product of two such e-erasure opera- 
tors is another e-erasure operator (a linear combination 
of the Ea's), the necessary and sufficient condition for 
erasure-correction becomes |13l 



{iQ\Ea\iQ) = {JQ\Ea\JQ) 

{iQ\Ea\JQ)^0 ioriy^j 



(2.9) 
(2.10) 



It results that an error-correcting code correcting t errors 
(at unknown positions) is equivalent to an e-erasure cor- 
recting code with e = 2t. This equivalence will be very 



useful in the following because the quantum erasure chan- 
nel is easier to treat using an entropic approach. Before 
coming to the information-theoretic analysis of quantum 
error-correcting codes (Section IV), let us first analyze 
classical error correction in terms of entropies. This will 
make the classical-quantum correspondence more trans- 
parent. 



III. ENTROPIC CONDITION FOR CLASSICAL 
ERROR/ERASURE CORRECTION 

Just like in the quantum case, one can define two 
classes of classical noisy channels, depending on the fact 
that the errors occur at known or unknown locations. In 
the former case, the located errors are called erasures, 
and an erasure-correcting code is such that, if e bits out 
of the n bits are "erased" , it is possible to recover the en- 
coded logical word from the n—e remaining bits only [E0| . 
In the latter case of classical codes capable of correcting 
t errors at unknown positions in codewords of size n, all 
the n bits of the corrupted codewords must be used in 
the decoding operation. Exactly as for quantum codes, it 
is easy to show that a classical code can correct t errors 
at unknown locations if and only if the same code can 
correct e = 2t erasures at known locations. The proof is 
as follows. Let us consider two codewords of length n, 
Wi and Wj, and two error strings, Gq and e^ (the bits in 
a codeword are flipped where the corresponding bits in 
the error string are equal to 1). To be able to recover t 
errors, we must have 



Wj © Ca 7^ Wj 
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(3.1) 



for any two codewords and for all possible error strings 
having t bits (or fewer) equal to one. Here, © is the bit- 
wise addition modulo 2 and ^ means that the two strings 
must differ by at least one bit. A classical code correcting 
t errors must therefore be such that the distance between 
any two codewords is larger than or equal to 2t+l, since 
the error strings Cq and Cb can have at most t bits equal 
to one, implying that ea © Cf, can have at most 2t bits 
equal to one. Now, in the case of codes capable of cor- 
recting e erasures, the positions of the bits equal to one 
in Ca and Cf, are identical, so that ec = ea ® e^ can have 
at most e (rather than 2e) bits equal to one and is there- 
fore another e-error string just as Ca or ei,. Thus, the 
condition for recovering e erasures is 



w. 



I Cc 7^ Wj 



(3.2) 



In other words, the distance between any two codewords 
must only be larger than or equal to e + 1. Obviously, 
Eq. (O) parallels Eqs. ( p!^^ , while Eq. (|^ parallels 
Eqs. (2^3-2. IC). The resulting equivalence e = 2t will be 



important for our concern because the entropic analysis 
is more adapted to erasure correction. 



Let us shortly describe coding in the case of a classical 
erasure channel Ig^l- We consider encoding as a classical 
channel whose input X is made of k logical bits and out- 
put Y is made of n physical bits (the codewords). We 
assume that the set of logical words Xi occur with prob- 
ability Pi , so that the entropy of the input X is 



H{X) = -J2p.^ogp, 



(3.3) 



The input X can be recorded (a classical variable can be 
"cloned") and thus compared with the output Y. As the 
encoding is reversible (it is a one-to-one mapping), the 
mutual entropy is conserved through encoding, that is 



/ = H{X:Y) = H{X:X) = H{X) 



(3.4) 



where / is defined as the mutual entropy (or informa- 
tion) between input and output that must be preserved 
in the classical erasure channel. Let us assume that Y 
is split into e erased bits. Ye, and n — e unchanged bits, 
Yu- (The position of the erased and unchanged bits is 
known.) The condition for classical erasure correction is 
clearly that the uncertainty of the input when the n ~ e 
unchanged bits are known vanishes, that is 



H{X\Yu) - 



(3.5) 



In other words, this means that the e bits can be erased 
without preventing the ability of inferring the input 
X from Yu without error. Since we have H{X\Y) — 
H{X\YeYu) = as a result of Eq. (3^), i.e., it is obvi- 
ously possible to infer X from Y = Y^Yu, we obtain the 
basic entropic condition for classical error correction 

H{X:Y,\Yu) = H{X\Yu) - H{X\YeYu) = (3.6) 

Physically this expresses that, conditionally on the n — e 
unchanged bits, no information about X is lost in the 
e erased bits. Classical coding works because the n — e 
unaffected bits contain the entire information / about X , 
that is 



HiX:Yu) = H{X:Y) - / , 

so that the e bits that are erased are "redundant" 
the chain rule for Shannon mutual entropies, 

H{X:YeYu) - HiX:Y^) + HiX:Y,\Yu) 



(3.7) 
Using 



(3.8) 



it is clear that Eq. (3.7) is satisfied if and only if the con- 
dition Eq. (3.6) is satisfied. In an erasure-correcting code, 
the k bits of information are thus distributed among the 
n bits of Y in such a way that condition Eq. (3.6) is satis- 
fied for any splitting of the n bits into e erased and n — e 
unchanged bits. The general classical entropy diagram 
corresponding to this situation is represented in Fig 0. 
The condition for erasure correction, Eq. (3.6), appears 



in this Figure as the vanishing entropy shared by X and 
Ye , but not by Y^ ■ 

FIG. 2. Entropy diagram for a classical erasure-correcting 
code. The input X stands for the logical bits, while the out- 
put Y (the codewords) is split into the erased bits Ve and 
the unchanged bits y„. The condition for erasure-correction 
is H{X:Y(^\Yu) ~ 0, that is the entire information must be 
found in the unchanged bits, H{X:Yu) — I. 

K X 




Let us briefly show that, for a classical code, it is im- 
possible that all the patterns of e "erasable" bits are inde- 
pendent oi X, i.e., do not contain some redundant infor- 
mation about X. (This feature turns out to be possible 
for a quantum code, as shown in Section IV.) Suppose 
that we could isolate two subparts of Y independent of 
X, that is two patterns of bits, say Yi and Y2, such that 



HiX:Yi) = H{X:Y2) = 



(3.9) 



Suppose also that, taken together, Yi and Y2 provide the 
entire information about X, that is 



H{X:YiY2) = I 



(3.10) 



IV. ENTROPIC CONDITION FOR QUANTUM 
ERROR/ERASURE CORRECTION 

A. Classical correspondence 

The above information-theoretic analysis can be 
straightforwardly applied to the case of a quantum 
erasure-correcting code. Here, the reference R plays the 
role of the input X, while Q (the quantum codewords) 
replaces the output Y. We also substitute the classi- 
cal notion of Shannon mutual entropy (information) be- 
tween X and Y with the quantum notion of von Neu- 
mann mutual entropy between R and Q, and use the 
extension to the quantum regime of the fundamental re- 
lations between Shannon entropies in a multipartite sys- 
tem |l^ , |l7| , pi| (see Appendix 0). First, the mutual en- 
tropy between the logical words L and R is conserved 
through encoding (since it is unitary), so that we have 



H{X:Yi)+H{X:Y2)>I 



(3.12) 



which is incompatible with Eq. (3.9) if / > 0. One of 
the subpart {Yi or Y2) must necessarily be correlated 
with X (have a non-vanishing mutual entropy with X) 
if the other one is independent of X. Some pattern of 
e "erasable" bits, including Yi or I2, will therefore be 
redundant (contain some information about X that is 
already in Yy) as a consequence of strong subadditivity. 



/, = S{R:Q) = S{R:L) 



(4.1) 



for the mutual entropy between the codewords Q and R. 
Here, Iq can be seen as the "quantum information" (the 
entanglement with R) which must be preserved in the 
quantum erasure channel. As before, we assume that Q 
is split into Qe (the e erased qubits) and Qu (the n — e 
unchanged qubits). Just like in the classical case, it is 
intuitively clear that entanglement is preserved at the 
condition that the total mutual entropy with R is found 
in the unaffected qubits, Qm that is 



S[R:Qu) - S{R:Q) = Ig 



(4.2) 



in analogy with Eq. ( p.7[ ). Using the chain rule for the 
quantum mutual entropy between R and Q = QeQu, 



This should be the case if we want to make a set of bits 
that fully determines X (such as Yu) out of pieces that 
are independent of X . We have 

H{X-yi) + H{X:Y2) = H{X:YiY2) + H{X:Yi:Y2) 

= I + H{Yi:Y2)-H{Yi:Y2\X) (3.11) 

Since the logical word X fully determines any bit of the 
codeword Y , we have H{Yi:Y2\X) = 0. Thus, the subad- 
ditivity of entropies, H{Yi:Y2) > 0, implies that 



S{R:QeQu) - S{R:Qu) + S{R:Qe\Qu) 



(4.3) 



we conclude that the condition for quantum erasure cor- 
rection is 



SiR■.Qe\Q^.) = 



(4.4) 



the straightforward analogue of Eq. (3.6). At this point, 
the parallel with classical erasure correction breaks down 
because of a peculiar property of quantum entropies. It is 
shown in Ref. p7| ] that the ternary mutual entropy of any 
entangled tripartite system in a pure state vanishes (see 
also Appendix ^). In the case of interest here, the tri- 
partite system RQ^Qu is in the pure state {tPrq), so that 
we have S {R: Qe'.Qu) — 0. As a consequence, we obtain 
from Eq. (4^) the basic entropic condition for quantum 
erasure correction 

S{R:Qe) - S{R:Qe\Qu) + S{R:Qe:Qu) = (4.5) 

Physically, this expresses that the "erased" part of the 
codewords Qe must be independent of the reference R. 
This is very different from the classical situation, where. 



in order to enable erasure correction, the erased bits must 
by construction be correlated with X. In other words, 
"classical redundancy" requires correlation of the e re- 
dundant bits with X, while "quantum redundancy" is 
achieved without correlating (or entangling) the erased 
qubits with R. We will show later on that the above 
entropic condition, Eq. ( |4.5| ), can be derived more rig- 
orously, using the property of strong subadditivity of 
quantum entropies and the entropic condition for perfect 
quantum error correction p9|,pl[. 



B. Quantum loss of a noisy channel 

As explained in Section II, we assume that the code- 
words sent on the quantum noisy channel suffer an ar- 
bitrary decoherence due to the environment E^ that is 
Uqe is an arbitrary unitary transformation. (We do 
not restrict ourselves to a quantum erasure channel for 
the moment.) After such an an arbitrary environment- 
induced decoherence, the joint sys tem R'Q'E' is in the 
state \ipR'QiE') given by Eq. (2.4). The corresponding 
quantum entropy diagram is represented in Fig. p^ (as 
mentioned earlier, the primes refer to the systems after 
decoherence) . 



S{E') + S{Q) - S{Q') 



(4.6) 



FIG. 3. Entropy diagram summarizing the entropic rela- 
tions between the entangled systems Q' (quantum channel), 
R' (reference), and E' (environment) after decoherence (see 
also Ref. Q). 




As shown in Ref. pT[ , it depends on three parame- 
ters, S — S{R') — S{R), the entropy of the reference R 
(which is also equal to the entropy of Q before decoher- 
ence), Se = S'(-B'), the entropy of the environment after 
decoherence, and^ 

L^S{R':E'\Q') 
= S{R'Q') + S{E'Q') - S{Q') - S{R'Q'E') 



the loss of the channel (following the terminology of 
Shannon theory p8| ) . The quantum loss L can be shown 
to be the analogue of the loss in a classical noisy channel, 
and thus can be written as a quantum conditional mu- 
tual entropy, i.e., the quantum mutual entropy between 
R' and E' , conditionally on Q' ||2l|. 

The loss L has a simple physical interpretation in the 
case of a classical noisy channel: it corresponds to the 
entropy of the input X of the channel conditional on its 
output Y , i.e., L = H{X\Y), thereby characterizing the 
unavoidable uncertainty in the decoding operation (when 
inferring the input from the corrupted output). Equiva- 
lently, it corresponds to the mutual entropy between the 
input and the environment, conditional on the output, 
i.e., L = H{X:E\Y). That is, for a given output, L 
measures the information about the input that has been 
irrecoverably lost in correlations with the environment. If 
X corresponds to encoded codewords and Y to corrupted 
ones due to a particular error source, the condition L = 
must be satisfied for the error-correcting code to preserve 
the codewords against classical noise ||18| . 

In Ref. |2l), it is shown that the same interpretation 
holds for the quantum loss L, substituting the classical 
notion of mutual information between X and E (condi- 
tional on Y) with the quantum notion of von Neumann 
mutual entropy between R' and E' (conditional on Q'). 
The reference R (= R') plays the role of the input X , 
while Q' replaces the output Y . Accordingly, it is ex- 
pected that a vanishing quantum loss corresponds to a 
situation where decoherence can be entirely eliminated 
using a quantum code. Indeed, 



L^Se + S- S{Q') ^ 



(4.7) 



is a necessary and sufficient condition for the existence 
of a perfect quantum error-correcting code, as proven re- 
cently by Schumacher and Nielsen [|9[. In Fig. ^, the 
entropy diagram of R'Q'E' is represented in the case 
where this condition is achieved. It appears that, when 
L = 0, the state of Q' becomes entangled separately 
with the environment ("bad" entanglement) and the ref- 
erence ("good" entanglement), allowing this "bad" en- 
tanglement to be transfered to an ancilla (the n — k check 
qubits) while recovering only the "good" one. This trans- 
fer of entanglement, requiring a local action on Q only 
(not on E), can be seen as a measurement of the error 



^ Since the total system R'Q'E' is in a pure state after de- 
coherence, i.e., S{R'Q'E') = 0, its Schmidt decomposition 
implies S{R'Q') = S(E') and S{E'Q') = S{R'), resulting in 
the last relation in Eq. (4.6). 



syndrome (the ancilla becoming entangled with E) leav- 
ing the original state intact. 

FIG. 4. Entanglement between Q' , R' , and E' in a loss- 
less (L = 0) quantum channel. The quantum system Q' is 
entangled "separately" with R! and E' (see also Ref. |2l|). 




The fact that Eq. (4.7) is a necessary condition can 
be understood simply by noticing that the loss can never 
decrease by processing Q' through a subsequent channel, 
for example in the decoding operation |^l|] . Denoting the 
loss after decoherence by Li and the overall loss (after 
decoherence and decoding) by L12, one has 



0<Li<Li 



(4.8) 



showing that Li = is necessary for having L12 — 0, 
that is for perfectly recovering decoherence by decoding. 
Unlike in the classical case, it is possible to rewrite the 
quantum loss as a function of E' and R' only, exploit- 
ing a purely quantum feature of entropies in a tripartite 
system. As mentioned earlier, an important consequence 
of S{R'Q' E') = is that the quantum ternary mutual 
entropy vanishes, that is 

S{R':E':Q') = S{R':E') - S{R':E'\Q') 

= S{R') + S{Q') + S{E') - S{R'Q') 
-S{R'E') - S{Q'E') + SiR'Q'E') 
= (4.9) 

As a result, the quantum loss can be expressed as 

L = S{R':E') (4.10) 

Therefore, a necessary and sufficient condition for perfect 
error correction is that the reference and the environment 
are statistically independent [L = 0). This condition re- 
lates entropies after decoherence, and thus allows us to 
check that, for a given code and after a specific interac- 
tion with the environment, decoherence can be recovered 
by decoding. As far as quantum coding is concerned, it is 
more useful to derive an entropic relation involving only 
the reference R and the codewords Q before unitary in- 
teraction with the environment, using some error model 



C. Upper bound on the quantum loss 

As before, we consider now an explicit error model in 
which the decoherence involves e qubits at know loca- 
tions, i.e., the case of e erasures. The component Qe (of 
e qubits) of the codeword interacts with E while the rest 
Qu (of n — e qubits) remains unchanged by the interac- 
tion. Accordingly, the unitary transformation describing 
such an error model is Urqe — '^r® 1q„ ® Uq^e- This 
results in the conservation rule for the mutual entropy 
(see Appendix ^), 



S{R':Q',E') = S{R:QeE) = 5(i?:Qe) 



(4.11) 



where we made use of the fact that E is initially in a pure 
state, i.e., S{E) = 0. This entropy can also be expressed 



S{R':Q',E') = S{R':E') + S{R':Q',\E') (4.12) 

by use of the chain rule for quantum mutual entropies. 
Using the strong subadditivity of quantum entropies, 

S{R':Q',\E') = S{R'E') + S{Q',E') 

-S{E') - S{R!Q'^E') > (4.13) 



and denoting by 



M = S{R:Qe) 



(4.14) 



the initial mutual entropy (or mutual entanglement) be- 
tween the reference R and the erased subpart Qe of the 



codeword, Eqs. (4.11) and (4.12) yield an upper bound 
on the loss L: 



0< L< M 



(4.15) 



Consequently, if the mutual entanglement M (initial mu- 
tual entropy between R and Qe) is zero. 



M = S{R:Qe) = 



(4.16) 



[cf Eq. (4.5)] 



then the loss L — S{R':E') vanishes, allowing for per- 
fect erasure correction. In other words, the statistical 
independence [M — 0) between the reference R and the 
erased part of the codeword Qe is a sufficient condition 
for perfect erasure correction, as anticipated in Eq. (|4.5|). 
Note that this condition must hold for any pattern of 
e erased qubits among the n qubits, a constraint which 
implies the quantum Singleton bound (see Section V). 

The physical content of the entropic condition, 
Eq. ( 4.16D , is the following. The reduced density matrix 
PbQc = '^'^Qu\'^b q){'4 'Rq\ obtained by tracing the state 
of RQ, i.e., Eq. ( |2.2| ), over Q„ (ignoring the n — e un- 
changed qubits) before decoherence must represent two 
independent systems: the k qubits of the reference R and 
the e erased qubits Qe of the quantum system. The latter 



e qubits can then be "erased" without interfering with R 
in the sense that the n — e remaining qubits retain all 
the entanglement with R. The general entropy diagram 
of the joint state of the system RQ = RQeQu before de- 
coherence is shown in Fig. (to be compared with Fig. 
for a classical code). 

FIG. 5. Entropy diagram for a quantum erasure-correcting 
code. It characterizes the combined system RQ = RQeQu 
before decoherence when the condition for perfect error cor- 
rection S{R:Qe) = is fulfilled. The two parameters are 
S{R) = k and S{Qe) = s. 




For a code to protect an arbitrary mutual entropy be- 
tween Q and R (or an arbitrary state for Q), the above 
condition M = must clearly be satisfied for the worst 
case in which the amplitudes ai in Eq. (^j) are all equal 
(|aip = 2^*^), that is in the case where Q and R "satu- 
rate" their entropy: 



SiQ) ^ S{R) 



(4.17) 



(Note that, since dq > d^, the entropy is limited by the 
size of the Hilbert space of R.) We will thus only consider 
this case in the following (it is important when deriving 
the Singleton bound on quantum codes). Because of the 
two constraints Eqs. ( 4.16| ) and (4.17), the ternary en- 
tropy diagram for RQeQu depends on a single unknown 
parameter, s = S{Qe), the entropy of the erased qubits^. 
In view of Fig. |5| we see that the e qubits of Qe are "su- 
perfluous" , as they do not yield any information about 
R (no mutual entanglement with R) and arc thus un- 
necessary for recovering the original logical state. In this 
sense, they constitute "redundant" quantum information 
since the total mutual entropy 2k is found between R 
and Qu- The unchanged qubits Qu are entangled sep- 
arately with the reference R ("useful" entanglement 2fc 
that must be preserved by the code) and with the erased 
qubits Qe ("useless" entanglement), so that any action 
on Qe due to an environment E can only transfer this 
"useless" entanglement to E but leaves the "useful" en- 



tanglement unchanged. Indeed, if the entropy diagram 
Fig. ^is achieved, then Eq. (4.15) implies that any inter- 
action between Qe and E necessarily results in an entropy 
diagram such as the one depicted in Fig. |j, where Q' is 
entangled separately with E' and R' {i.e., L = 0), guar- 
anteeing that one can undo decoherence by applying an 
appropriate decoding. 



D. Example 

As an illustration, we show in Fig. |^ the entropy di- 
agram in the case of a 5 qubit code {n — 5) encoding 
k = 1 logical qubit with t = 1, i.e., allowing up to e = 2 
erasures H,g| . The full mutual entanglement of 2 bits is 
found between R and Qu, while the 2 erased qubits Qe 
are independent of R. We have S{R) = 1, S{Qu) = 3, 
and S{Qe) = 2, so that each subsystem has the maximum 
allowed entropy for its Hilbert space {R, Qu, and Qe are 
made of 1, 3, and 2 qubits, respectively). Note that the 
4 qubit code {n — 4) encoding k — 2 logical qubits and 
correcting e = 1 erasure jl^,|l^ corresponds in fact to 
the same entropy diagram with R playing the role of Qe 
and conversely. Indeed, R has then an entropy of 2 bits 
and shares a mutual entropy of 4 bits with Qu (which 
then contains the full information about the 2 encoded 
qubits). This mutual entanglement is preserved against 
erasure of 1 qubit since Qe is independent of R. 

FIG. 6. Quantum entropy diagram of the combined sys- 
tem RQeQu before decoherence for the 5 bit quantum code 
(n = 5, fc = l, e = 2) |,|. 




Let us finally compare this "quantum redundancy" in 
the 5 qubits code with the entropic diagram character- 
izing a classical code. As explained in Section III, in 
a classical code the information (fc bits) is distributed 
among the n bits which are then correlated with the in- 
put X in a specific way [so that Eq. ( p.6[ ) is satisfied]. 
Ignoring the n~ e unchanged bits (y„) leaves e bits (Ye) 



■^The entropy diagram for a general tripartite system in 
pure state depends on three parameters. 



that are redundant [the total information is in y„, as im- 
pUed by Eq. ( |3.7| )], but correlated with the input X, in 
contrast with the quantum case. The entropy diagram 
corresponding to a simple classical code is illustrated in 
Fig. M (to be compared with Fig. 0). We consider a simple 
linear code with n = 5, k = 2, e = 2, defined in Ref. p3] 



00- 


-^ 00000 


01- 


-^01110 


10- 


-^ 10101 


11 - 


-^ lion 



(4.18) 



such that the distance between any two codewords is 
3 or larger. We assume that the first and the last bit 
are erased (i^e), the three other ones being unchanged 
(Yu), and show the entropy diagram in the case where 
the logical words 00 to 11 are equiprobable [the entropy 
in Eq. (3.3) is maximum]. The full information is found 
in Yu, H{X:Yu) = 2 bits, but the erased bits are par- 
tially correlated with X, H{X:Ye) = 1 bit. Classical re- 
dundancy necessarily implies that the erased bits contain 
part of the information that is duplicated. 

FIG. 7. Classical entropy diagram of XYeYu for the n = 5, 
k = 2, e — 2 classical linear code defined in the text. 




The situation is thus quite different in a quantum 
code: E q. (3.6) is replaced by its quantum counterpart 
Eq. (14), with the same physical interpretation, b ut the 
latter equation then implies the simpler condition ( 4.16| ) 
as a consequence of the property that the quantum 
ternary mutual entropy vanishes for a pure state. Such a 
possibility to achieve "weak" cloning through coding (in 
the sense that the full information is in Qu without cor- 
relating Qe with the reference R) is purely quantum and 
suggests an interesting interpretation of quantum coding, 
as explained in Section VI. 



V. SINGLETON BOUND ON QUANTUM CODES 

The above entropic considerations provide a simple 
way to derive the quantum analogue of the Singleton 
bound on error-correcting codes, obtained recently by 



Knill and Lafiamme ||ll|]. For a classical code, the Sin- 
gleton bound (see, e.g., |2^) states that the number of 
logical bits k than can be encoded in a code of length n 
recovering e erasures is such that 



k < n — e 



(5.1) 



Of course, for a classical code recovering t errors (at un- 
known locations), the Singleton bound becomes 



k <n-2t 



(5.2) 



as a consequence of the equivalence between codes cor- 
recting t errors and e = 2t erasures. In order to de- 
rive the quantum analogue of this bound, we consider 
the joint state of the system RQ — RQeQu before de- 
coherence. For a quantum code to protect an arbitrary 
entanglement between Q and R, the entropic condition 
M = S{R:Qe) = must is satisfied for the worst case of 
maximum entanglement, that is in the case where Q and 
R "saturate" their entropy S{Q) — S{R) = k. Assume 
for the moment that the code is such that S{Qe) — e, 
i.e. that the erased qubits have the maximum entropy al- 
lowed by the dimension of the Hilbert space of Q^. Then, 
the condition Af = is clearly satisfied if tracing over the 
n — e qubits associated with Qu yields a reduced density 
matrix for RQe that saturates its quantum entropy 



S{RQe) = S{R) + S{Qe) - S{R:Qe 



(5.3) 



This corresponds to the case s = e in Fig. ||. Since 
RQeQu is in a pure state [i.e., S (RQeQu) — 0], one has 
S{RQe) — S{Qu) by Schmidt decomposition. Expressing 
that the quantum entropy of Qu is bounded from above 
by the logarithm of the dimension of its Hilbert space, 
that is S{Qu) l£ n — e, one gets the inequality 



k < n — 2e 



(5.4) 



which is the Singleton bound for quantum erasure- 
correcting codes. Making use of the equivalence between 
codes correcting errors of t qubits and the erasure of 
e — 2t qubits, we get the Singleton bound for quantum 
error-correcting codes (proven in Ref. |1^ for fc = t = 1) 



k<n-At 



(5.5) 



This condition must be satisfied by any quantum code 
(including degenerate codes). Mathematically, Eq. (5.4) 
expresses thus that it is necessary to trace over at least 
half of the n + k qubits constituting the total entangled 
state \tpRQ) in order to open the possibility of having 
k + e independent remaining qubits (that is which sat- 
urate their entropy), thereby allowing error correction. 
Eq. (5.4) suggests an interpretation of quantum coding 
in terms of a "weak" cloning, as explained in the next 
Section. 

The above derivation was based on the assumption 
that the erased qubits have a maximum entropy, i.e.. 



S{Qe) — e. This is true for example in the case of the 5 
qubit code shown in Fig. 0. However, we need to prove 
Eq. { ^A\ j in full generality, without recourse to this as- 
sumption. In general, it is possible to have S{R:Qe) = 
with S(Qe) < e; this is the case for example when one 
(or more) of the physical qubits is always (non-optimal 
code). Suppose that the condition for erasure correction 
S{R:Qe) = is satisfied for some pattern of e erased 
qubits, so that the remaining part of the codeword Qu 
retains the "full" entanglement, S{R:Qu) = 2fc. The 
central point in deriving the quantum Singleton bound is 
that this entropic condition must be fulfilled for any pat- 
tern of e erased qubits among the n qubits. Therefore, 
one can choose for example another pattern of e qubits 
within the n — e qubits that constitute Qu, and check 
that they have also a vanishing mutual entropy with R.^ 
Let us denote the e erased qubits in this second check by 
Qe, so that Qu is divided into Q'^ and Q* (the n ~ 2e 
remaining qubits) as shown in Fig. g|. The correspond- 
ing entropic condition is thus S{R:Q'^) — 0. Conversely, 
the unchanged qubits Q'^ = QeQ* in this second check 
must also retain the full mutual entanglement with _R, 
i.e., S{R:Q'^) = 2k. This implies that, while the e qubits 
of Qe are independent of R, they must recover the to- 
tal mutual entanglement 2k with R when supplemented 
only with the n — 2e qubits of Q* . These two opposite 
constraints must be satisfied simultaneously, which gives 
rise to the quantum Singleton bound. 



S{Q',Q*) < 5(Q'J + S{Q* 



implies the inequality 



k + S{Qe)~S{Q'J<S{Q* 



(5.7) 



(5.8) 



By the same token, given the independence between Qe 
and R, we can calculate the entropy of Q'^, 



SiQ'J = SiQeQ*) 

- SiRQ'e) 

^SiR) + S{Q'J-SiR:Q'J 
^k + SiQ',) 



and make use of subadditivity 

SiQeQ*)<S{Qe) + S{Q*) 



to obtain 



k + SiQ',)-S{Qe)<S{Q*) 



(5.9) 



(5.10) 



(5.11) 



Finally, combining Eqs. (5.S) and (5.11) provides a lower 
bound for SiQ*): 



FIG. 8. Schematic representation of the two different split- 
tings of Q into QeQu or Q'eQ'u which are used in the derivation 
of the quantum Singleton bound. 
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In order to prove this bound, we first calculate a 
lower bound on the entropy of Q* . Using the fact that 
RQ — RQeQ'eQ* is in a pure state and the independence 
between Qe and i?, we have 

S{Qu) = S{Q',Q*) 

= S{RQe) 

= SiR) + S{Qe)-SiR:Qe) 

^k + S{Qe) (5.6) 

Then, the property of subadditivity of quantum entropies 



k<S{Q* 



(5.12) 



This bound is equivalent to S{QeQ'e\R) > 0, and has 
the following interpretation. Even though Qe and Q^ are 
both independent of R, the combined system QeQ'e will 
generally be entangled with R (with a mutual entropy 
between and 2k). However, in contrast with the entan- 
glement between Q„ (or Q^) and R, the entropy of QeQ'e 
conditional on R cannot become negative because of the 
opposite constraints on S{Qe) — S{Q'e) from Eqs. ( |5.8| ) 
and (5.11). The quantum Singleton bound is obtained 
simply by noticing that S{Q*) is bounded from above by 
the dimension of the Hilbert space of Q*, that is 



S{Q*) <n-2e 



(5.13) 



The latter equation together with Eq. (5.12) completes 
the proof of Eqs. (5.4) and (O). 



This implies the simple constraint that the dimension of Qu 
must be larger that the dimension of Qe, that is e < n — e. 
This inequality, i.e., the Singleton bound for k = 1, also re- 
sults straightforwardly from the no-cloning theorem (see Sec- 
tion VI). 
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VI. DISCUSSION AND CONCLUSION 

Before concluding, let us discuss the relation between 
the quantum no-cloning theorem and quantum erasure- 
correcting codes. The main point is to note that, if we 
can erase e qubits while being able to recover the code- 
word, it means that the n—e remaining qubits contain all 
the information, so that the e qubits apparently contain a 
(partial) duplication of the logical word. Clearly, it is for- 
bidden to erase half (or more) of the n qubits, since then 
the two halves of the codeword could be mapped on the 
logical word, enabling quantum cloning. Thus, one must 
have n — 2e > (a constraint equivalent to the condition 
that the dimension of Qu must exceed the dimension of 
Qe)- However, Eq. (5.4) is actually more restrictive, im- 
plying that it is possible to quantify the impossibility of 
cloning (to be more precise than a yes-or-no theorem). 
We will see that the Singleton bound expresses that a 
"weak" cloning is allowed, up to a certain extent. Let 
us define the number of clones (the fractional number of 
copies of the logical word) as: 



N, 



(6.1) 



where the n—e qubits constitute the "original" (necessary 
to fully recover the logical state) while the e erased qubits 
make the partial'* clone. It is easy to see from Eq. (5.4) 
that the fractional number of clones is restricted to the 
range 



< 7Vc< 



(6.2) 



This somehow extends the standard no-cloning theorem 
to "weak" {Nc < 1) cloning. The no-cloning theorem fl^ ] 
states that it is forbidden to make one full clone, i.e., 
Nc y^ 1, while Eq. ( |6.2| ) provides an upper bound on 
weak cloning. In the limiting case where n = k, the num- 
ber of clones is strictly zero. This simply means that, if 
the codewords span the full 2"-dimensional Hilbert space 
{i.e., if no coding is actually used), then no cloning at all 
is achieved {Nc = 0). The same is obviously true for a 
classical code, since, using Eq. (5J), the equivalent con- 
dition on "weak" cloning is < Nc < {n — k)/k. When 



quantum coding uses only part of the Hilbert space, 
i.e., the space of codewords is some 2 -dimensional sub- 
space of the full space {k < n), the logical states may 
then be viewed as partially cloned by the encoding pro- 
cess, the fractional number of clones being limited to 
{n — k)/{n + k). The latter increases as a smaller sub- 
space is used {k decreases), and tends to one (full cloning) 
when n/k -^ cx). The case fc = corresponds to perfect 
cloning of a fixed {i.e., non-arbitrary) pure state. There- 
fore, whatever the apparent "replication factor" n/k of 
the logical words achieved by the encoding process, the 
allowed number of clones iVc < 1 (for a non- vanishing k). 
For a classical code, however, no such limit exists on the 
number of clones, as Nc — > n/k when n/k — > cxd. 



We have shown that some new insight into quantum 
coding can be gained by use of an information-theoretic 
approach paralleling the one used to describe classical 
coding. Such an analysis displays explicitly the similar- 
ities between classical and quantum codes, but also em- 
phasizes the major differences. The entropic condition 
for a quantum erasure-correcting code is that the quan- 
tum mutual entropy between a reference and the erased 
part of the codeword is vanishing prior to decoherence. 
Such a statistical independence between the reference 
and the erased qubits (interacting with the environment) 
guarantees that the entanglement of the logical word with 
respect to this reference is preserved by the quantum 
code. This is to be compared with the corresponding 
entropic condition for a classical erasure-correcting code, 
i.e., that the mutual information between the logical bits 
and the erased bits of the codewords, conditional on the 
remaining unchanged bits of the codewords, is vanishing. 
Such a classical condition, however, does not imply that 
the erased bits are independent of the logical bits. On the 
contrary, there must be correlations between them, and 
this duplication (or "cloning" ) of classical information is 
at the heart of classical codes. Such a classical redun- 
dancy has no quantum counterpart, as a consequence of 
the purely quantum property that the ternary mutual 
entropy vanishes for any entangled tripartite system in a 
pure state. In a quantum code, only a "weak" cloning is 



*This concept of "partial" cloning is unrelated to the notion 
of "approximate" cloning introduced in Ref. [M. There, a 
universal quantum-cloning machine is used that has two out- 
puts being an approximate copy of the input. This can be 
viewed as two channels sharing the same input but necessar- 
ily characterized both by a non-vanishing quantum loss, i.e., 
the fidelity of both copies is not one. In our case, we have 
two lossless channels: L —> Q„ and L -^ Q'^. However, the 
outputs unavoidably share a common piece Q* which cannot 
be reduced to zero for S{R:Qu) ~ '2k and S{R:Q'u) = 2k to 
hold simultaneously. 
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achieved, up to the extent allowed by the quantum Sin- 
gleton bound, so that the erased qubits are unentangled 
with the reference although the entire codeword remains 
entangled with it. This reflects a major difference be- 
tween classical and quantum coding. 



Y, that is the information about X that is conveyed by 
Y, or the decrease of the entropy of X due to knowledge 
of Y (or conversely): 

H{X:Y) = H{X) - H{X\Y) = H{Y) - H(Y\X) (A4) 
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APPENDIX A: INFORMATION-THEORETICAL 
FRAMEWORK FOR QUANTUM ENTROPIES 

Classical information theory is centered on Shannon 
entropies (see, e.g., |18 2^]). A random variable X, dis- 
tributed according to the probability distribution pi, is 
characterized by the Shannon entropy 



H{X) = -Y,pdogp, 



(Al) 



The Shannon entropy H{X) measures the uncertainty oi 
X (it vanishes if the distribution is peaked, i.e., if the 
value of X is perfectly known). When considering two 
random variables X and Y , described in general by the 
joint probability distribution Pij, one can define several 
entropies. First, one has the joint entropy H{XY), based 
on pi,j in analogy with Eq. ( |Al|) , which reflects the un- 
certainty of X and Y . Second, one defines the entropy of 
X conditional on Y , that is the entropy of X when Y is 
known (averaged over Y), 



H{X\Y) = -Y,p,,,\ogp,\, 

= H{XY) - H{Y) 



(A2) 



based on p^\j — pij/pj, the conditional probability of 
i knowing j. The equivalent definition holds for the 
conditional entropy of Y knowing X, i.e., H{Y\X) = 
H{XY) — H{X). Finally, one defines the mutual entropy 
between X and Y as 



i/(A:r) = -^p,,,log 



Pi:3 



H{X) + H{Y) -H{XY) 



(A3) 



where pi-^ = piPj /pij is the mutual probability of i and j. 
It plays the role of a mutual information between X and 



Consider now the information-theoretical description 
of a bipartite quantum system XY, where X and Y cor- 
respond to two quantum variables or degrees of freedom 
{e.g., the z-component of a spin-1/2 particle). It is shown 
in Ref. [l^Jl^l that a quantum information-theoretical 
formalism that parallels Shannon construction can be de- 
fined that is based on the von Neumann entropy, where 
probability distributions are replaced by density matri- 
ces, and averages are changed into quantum expectation 
values. We have for the quantum (von Neumann) entropy 
of A, 



S{X) = -Trxipx logpx) 



(A5) 



where px is the density matrix characterizing the state 
of X. The density matrix px is obtained by a partial 
trace over the Y variable, i.e., px = Try(pjcY), where 
pxY is the joint density matrix describing XY. A similar 
definition holds for S{Y), based on py. One defines then 
the quantum (von Neumann) conditional entropy, 



SiX\Y) = -TlxYipXYlogpxlY) 
= S{XY) ~ S{Y) 



(A6) 



based on the conditional density matrix pxiY (defined 
in p6| , p7t ). The latter plays the role of a "quantum" con- 
ditional probability, and witnesses the appearance of non- 
classical correlations in the case of quantum entangled 
variables X and Y. Indeed, it can be shown that an eigen- 
value oi px\Y can exceed one, and, consequently, that the 
quantum conditional entropy S{X\Y) can be negative, a 
fact related to quantum non-separability (see ||lq , [l7| , P5|] ) . 
For example, if X and Y represent two entangled quan- 
tum bits in a Bell state, we have S{X\Y) = S{Y\X) = 
— 1 bit. Since negative conditional entropies are forbid- 
den in Shannon theory, a negative value of S{X\Y) ob- 
viously implies quantum non-separability, the converse 
being not true. On the other hand, if pxY describes a 
mixture of orthogonal product states (that is a classi- 
cal situation), px\Y is then a diagonal matrix with the 
Pi\j^s on its diagonal, and Eq. ( A6) reduces to its classical 
counterpart, Eq. (A2). The above definition, Eq. (|A6|), 
must therefore be viewed as a quantum extension of the 
Shannon conditional entropy in a way that incorporates 
quantum entanglement, while including the classical con- 
ditional entropy as a special case. 

According to this, it is natural to define a quantum 
(von Neumann) mutual entropy, 
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S{X:Y) = -TTxYipxY\ogpx:Y) 
= S{X) + S{Y) - S{XY) 
= S{X)-S{X\Y)^S{Y)-S{Y\X) (A7) 

based on the mutual density matrix px-.v (defined 
in ||lq,0| ) . The interpretation is the same as in Shannon 
information theory, and S{X:Y) is a symmetric quan- 
tity. Subadditivity of quantum entropies, i.e., S{XY) < 
S{X) + S{Y), implies that S{X:Y) > 0, just as for Shan- 
non mutual entropies. However, in the case of quantum 
entangled variables, S{X:Y) can reach twice the maxi- 
mum allowed value in Shannon theory ||lq,[17| : 



tite system XY undergoes a unitary transformation of 
the form Ux ® Uy- Indeed, if 



S{X:Y) <2mm[S{X),S{Y)] 



(A8) 



For instance, we have S{X:Y) — 2 bits between the mem- 
bers of an EPR pair. Note that S{X:Y) is not a measure 
of entanglement in the sense that it can be non-zero for 
classical (separable) mixtures. It does not necessarily 
exceed the classical upper bound, nim[S{X),S{Y)], for 
quantum entangled systems. Thus, S{X:Y) is rather a 
quantum mechanical extension of the usual Shannon mu- 
tual entropy, which measures quantum as well as classical 
correlations. Nevertheless, it plays an important role in 
the information-theoretic description of quantum chan- 
nel and error correction, as emphasized throughout this 
paper (see also [pl[). 

Consider a quantum system X entangled with a ref- 
erence R, so that the joint system RQ is in the pure 
state X^i VP^N-R)!*-^)- '^^'^ system X is sent through a 
quantum channel (which does not act on R). It is easy 
to see that, if the channel (including error-correction) is 
"perfect", i.e., if the output Y ends up in a joint pure 
state (together with R) such that the quantum mutual 
entropy with R is conserved, then an arbitrary quantum 
state is preserved in the channel^. Indeed, as R is un- 
changed, any joint state of RY that is characterized by 
S{R:Y) = S{R:X) and S{RY) = S{RX) = is neces- 
sarily of the form J^i \/Pi\iR){U\ix)) where [/ is a fixed 
unitary transformation (it does not depend on the pi's). 
Thus, up to a given change of basis, the output Y is in 
the same entangled state with R. Projecting R onto any 
pure state shows that the corresponding pure state of X 
(the "relative" state) has been preserved, i.e., Y ends up 
in the same state. 

Another important property of the quantum mutual 
entropy S{X:Y) is that it is conserved when the bipar- 



p'xY = (Ux ® Uy)pxy{Ux ® UyY 



(A9) 



then p^ = Try/(p^y) = UxPxUx and similarly for p'y, 
so that 



S{X':Y') ^ S{X:Y) 



(AlO) 



follows from Eq. (A7) together with the conservation of 
von Neumann entropy under a unitary transformation. 
In particular, any entangled system XY that undergoes 
a local operation separately on X and Y retains its ini- 
tial entanglement between X and Y , i.e., S{X:Y) is con- 
served. This property is useful in the context of quantum 
channels and quantum error correction. 

The quantum information-theoretical formalism de- 
fined above can be generalized to multipartite systems, 
in analogy to the Shannon construction ||l^,|l^ . The def- 
inition of the conditional (and mutual) density matrices 
provides grounds for the quantum extension of the usual 
algebraic relations between Shannon entropies (see, e.g., 
p8| , p0| ). The resulting framework for quantum informa- 
tion theory goes beyond classical correlations, i.e., ac- 
counts for situation where n quantum variables are en- 
tangled, by allowing conditional entropies to be negative. 
Of course, it also includes Shannon theory as a special 
case. Consider, for example, a tripartite quantum system 
XY Z . First, one can write the chain rule for quantum 
entropies, 

S{XYZ) = S{X) + S{Y\X) + S{Z\XY) (All) 

One can also define the von Neumann conditional mutual 
entropy, 

s{x-y\z) = s{x\z) - s{x\YZ) 

= S{X\Z) + S{Y\Z) - S{XY\Z) 

= S{XZ) + S{YZ) - S{Z) - S{XYZ) (A12) 

which reflects the quantum mutual entropy between X 
and Y, when Z is known. Eq. (A12) follows the defi- 
nition of a mutual entropy, Eq. (A7), but with all en- 
tropies being conditional on Z . Just like with classical 
entropies, the property of strong subadditivity of quan- 
tum entropies holds, that is S{X:Y\Z) > 0. Conditional 



^The classical analogue of this property is intuitive. If the 
input X of a classical channel is copied into a memory M 
(so that M is thus perfectly correlated with X), the correla- 
tion between the output Y and the memory M reflects the 
"quality" of the channel. In particular, if the mutual entropy 
between Y and M is equal to that between X and M, then 
the classical channel is perfect (lossless). 
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mutual entropies are also used in the quantum analogue 
of the chain rules for mutual entropies, that is 



S{X:YZ) = S{X:Z) + S{X:Y\Z) 



(A13) 



Finally, the relation between conditional and mutual en- 
tropies, S{X) = S{X\Z) + S{X:Z) can be extended to a 
tripartite system, that is 



S{X:Y) = S{X:Y\Z) + S{X:Y:Z) 



(A14) 



so that we can split S{X:Y) into a conditional piece and 
a mutual piece with Z. The latter piece, S{X:Y:Z), char- 
acterizes therefore the ternary mutual entropy, i.e., that 
piece of the mutual entropy between X and Y that is also 
shared by Z. All these relations between entropies can be 
understood very easily using Venn diagrams |16|Jl7|j2l[| . 

Finally, an important property of quantum entropies 
which has no classical counterpart, is that, for any entan- 
gled tripartite system XYZ in a pure state, the ternary 
mutual entropy vanishes p^ , i.e., 

S{X:Y:Z) = S{X) + S{Y) + S{Z) - S{XY) 
-S{XZ) - S{YZ) + S{XYZ) 
= (A15) 

This results from the fact that S{XYZ) = implies 
SiXY) = S{Z), S{XZ) = S{Y), and S{YZ) = S{X), as 
a consequence of the Schmidt decomposition of the state 
of XYZ. 
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